Nonverbal Signals, Multimodal Feedback

Nonverbal Signals, Multimodal Feedback

Let them Cry : A baby's cry causes analysis

using a wearable device and an application

Let them Cry : A baby's cry causes analysis

using a wearable device and an application

Let them Cry : A baby's cry causes analysis using a wearable device and an application

Problem

Problem
Problem

91.5% of caregivers said they felt stressed by their baby’s crying.

91.5% of caregivers said they felt stressed by their baby’s crying.

91.5% of caregivers said they felt stressed by their baby’s crying.

A baby’s cry is their first and only way to communicate.

However, many caregivers don’t know why the baby cries, making it hard to respond properly.

A baby’s cry is their first and only way to communicate.

However, many caregivers don’t know why the baby cries, making it hard to respond properly.

A baby’s cry is their first and only way to communicate.

However, many caregivers don’t know why the baby cries, making it hard to respond properly.

Solution

Solution
Solution

Confidence in Care

Confidence in Care

Confidence in Care

The device detects cries and explains the reason, giving caregivers clarity.

The device detects cries and explains the reason, giving caregivers clarity.

The device detects cries and explains the reason, giving caregivers clarity.

Practical Support

Practical Support

Practical Support

There's a change!

There's a change!

There's a change!

His crying pattern has changed, but that's okay, he's thriving!

His crying pattern has changed, but that's okay, he's thriving!

His crying pattern has changed, but that's okay, he's thriving!

Cry cycle

Cry cycle

Cry cycle

Cry pattern changes

Cry pattern changes

Cry pattern changes

Typical

Typical

Typical

Typical

Typical

Typical

3 Hours

3 Hours

3 Hours

8 Days

8 Days

8 Days

Hungry

Hungry

Hungry

Discomfort

Discomfort

Discomfort

Sleepy

Sleepy

Sleepy

Bored

Bored

Bored

찝찝함

찝찝함

찝찝함

Feeding them cold formula works well.

Feeding them cold formula works well.

Feeding them cold formula works well.

Note-taking keywords

Note-taking keywords

Note-taking keywords

Adjusting the feeding experience

Adjusting the feeding experience

Adjusting the feeding experience

61%

61%

61%

39%

39%

39%

Adjusting the feeding experience

Adjusting the feeding experience

Adjusting the feeding experience

5 minutes on the back

5 minutes on the back

5 minutes on the back

Low temperature formula

Low temperature formula

Low temperature formula

Split feeding

Split feeding

Split feeding

21 cases

21 cases

21 cases

14 cases

14 cases

14 cases

8 cases

8 cases

8 cases

2 cases

2 cases

2 cases

Last week

Last week

Last week

The app provides scientific analysis and solutions for daily care.

The app provides scientific analysis and solutions for daily care.

The app provides scientific analysis and solutions for daily care.

Pattern Insights

Pattern Insights

Pattern Insights

Tracking crying patterns helps caregivers understand their baby and gain confidence.

Tracking crying patterns helps caregivers understand their baby and gain confidence.

Tracking crying patterns helps caregivers understand their baby and gain confidence.

Problem

Problem
Problem

Unexplained crying can overwhelm first-time caregivers.

Unexplained crying can overwhelm first-time caregivers.

Unexplained crying can overwhelm first-time caregivers.

91.5%

91.5%

91.5%

The baby's crying is stressing me out.

The baby's crying is stressing me out.

The baby's crying is stressing me out.

85.6%

85.6%

85.6%

I've had a baby cry because I didn't get what I wanted.

I've had a baby cry because I didn't get what I wanted.

I've had a baby cry because I didn't get what I wanted.

*2024.01.01~01.10 Parenting cafe survey for parents under 16 months old

*2024.01.01~01.10 Parenting cafe survey for parents under 16 months old

*2024.01.01~01.10 Parenting cafe survey for parents under 16 months old

Literature Research

Literature Research
Literature Research

Parents feel most stressed by crying in the first 6 weeks.

Parents feel most stressed by crying in the first 6 weeks.

Parents feel most stressed by crying in the first 6 weeks.

“Crying peaks around 6 weeks of age, which overlaps with the period when

parents report the greatest stress in childcare.”

“Crying peaks around 6 weeks of age, which overlaps with the period when

parents report the greatest stress in childcare.”

“Crying peaks around 6 weeks of age, which overlaps with the period when

parents report the greatest stress in childcare.”

Vermillet, A., Tølbøll, K., Litsis Mizan, S., C. Skewes, J., & Parsons, C. E. (2022). Crying in the first 12 months of life:

A systematic review and meta‐analysis of cross‐country parent‐reported data and modeling of the “cry curve.” Child Development, 93(4), 1201–1222. https://doi.org/10.1111/cdev.13760

Vermillet, A., Tølbøll, K., Litsis Mizan, S., C. Skewes, J., & Parsons, C. E. (2022). Crying in the first 12 months of life:

A systematic review and meta‐analysis of cross‐country parent‐reported data and modeling of the “cry curve.” Child Development, 93(4), 1201–1222. https://doi.org/10.1111/cdev.13760

Vermillet, A., Tølbøll, K., Litsis Mizan, S., C. Skewes, J., & Parsons, C. E. (2022). Crying in the first 12 months of life:

A systematic review and meta‐analysis of cross‐country parent‐reported data and modeling of the “cry curve.” Child Development, 93(4), 1201–1222. https://doi.org/10.1111/cdev.13760

Contextual Inquiry

Contextual Inquiry
Contextual Inquiry

Caregivers seek to understand crying to connect & communicate better.

Caregivers seek to understand crying to connect & communicate better.

Caregivers seek to understand crying to connect & communicate better.

Competitor Analysis

Competitor Analysis
Competitor Analysis

Existing cry-analysis devices are cumbersome and not intuitive.

Existing cry-analysis devices are cumbersome and not intuitive.

Existing cry-analysis devices are cumbersome and not intuitive.

Why Cry Plus

Why Cry Plus

Why Cry Plus

Result check

Result check

Result check

Name

Name

Name

Image

Image

Image

Result display

Result display

Result display

Doesn’t analyze specific reasons

Doesn’t analyze specific reasons

Doesn’t analyze specific reasons

LED light + small text

LED light + small text

LED light + small text

LED light + small text

LED light + small text

LED light + small text

LED light + icon

LED light + icon

LED light + icon

Not easy to carry around

Not easy to carry around

Not easy to carry around

Not easy to carry around

Not easy to carry around

Not easy to carry around

Why Cry Mini

Why Cry Mini

Why Cry Mini

Biloop Cry Translator

Biloop Cry Translator

Biloop Cry Translator

Not intuitive

& slow

Not intuitive

& slow

Not intuitive

& slow

Not analyzing specific reasons

Not analyzing specific reasons

Not analyzing specific reasons

Poor

portability

Poor

portability

Poor

portability

How might we..

How might we..
How might we..

.. reduce their stress?

.. reduce their stress?

.. reduce their stress?

.. help them communicate with baby better?

.. help them communicate with baby better?

.. help them communicate with baby better?

UX Goal

UX Goal
UX Goal

Aiming to ease parenting anxiety and offer real help.

Aiming to ease parenting anxiety and offer real help.

Aiming to ease parenting anxiety and offer real help.

😱

😱

😱

Ease parental anxiety

Ease parental anxiety

Ease parental anxiety

Analyze baby crying causes & provide clear feedback

Analyze baby crying causes & provide clear feedback

Analyze baby crying causes & provide clear feedback

💨

💨

💨

Rapid response

Rapid response

Rapid response

Intuitive and easy interface for fast response

Intuitive and easy interface for fast response

Intuitive and easy interface for fast response

📖

📖

📖

Continuous learning

& adaptation

Continuous learning

& adaptation

Continuous learning

& adaptation

Personalized solutions that are increasingly accurate to your baby's characteristics

Personalized solutions that are increasingly accurate to your baby's characteristics

Personalized solutions that are increasingly accurate to your baby's characteristics

Opporturnity

Opporturnity
Opporturnity

Integrating services with wearable devices.

Integrating services with wearable devices.

Integrating services with wearable devices.

*Survey Conducted(2024.02), n=22

*Survey Conducted(2024.02), n=22

*Survey Conducted(2024.02), n=22

Q. What are some things you try to keep in mind when parenting a newborn?

Q. What are some things you try to keep in mind when parenting a newborn?

Q. What are some things you try to keep in mind when parenting a newborn?

Keeping an eye on the baby 89.5%

Keeping an eye on the baby 89.5%

Keeping an eye on the baby 89.5%

Importance of real-time monitoring

Importance of real-time monitoring

Importance of real-time monitoring

Respond quickly to small incidents 65.5%

Respond quickly to small incidents 65.5%

Respond quickly to small incidents 65.5%

Instant alerts and quick feedback

Instant alerts and quick feedback

Instant alerts and quick feedback

Being close to baby 75.4%

Being close to baby 75.4%

Being close to baby 75.4%

Easy accessibility

Easy accessibility

Easy accessibility

Even while juggling other chores,

using a wearable and an app together helps you respond more quickly and proactively.

Even while juggling other chores,

using a wearable and an app together helps you respond more quickly and proactively.

Even while juggling other chores,

using a wearable and an app together helps you respond more quickly and proactively.

Appropriateness Evaluation

Appropriateness Evaluation
Appropriateness Evaluation

The wearable device should be a necklace type.

The wearable device should be a necklace type.

The wearable device should be a necklace type.

[ Product Sketch ]

[ Product Sketch ]
[ Product Sketch ]

Wearable Device Display

Wearable Device Display
Wearable Device Display

Hands-free care, real-time connection.

Hands-free care, real-time connection.

Hands-free care, real-time connection.

Hands-free

Hands-free

Hands-free

Design the display to be in a position

where users can see it without the use of both hands

Design the display to be in a position

where users can see it without the use of both hands

Design the display to be in a position

where users can see it without the use of both hands

Compact device

Compact device

Compact device

Smallest display size that stays compact

and doesn't interfere with readability

Smallest display size that stays compact

and doesn't interfere with readability

Smallest display size that stays compact

and doesn't interfere with readability

*Wearable device displays

*Wearable device displays

*Wearable device displays

Annoyed

Annoyed

Annoyed

Annoyed

Annoyed

Annoyed

For caregivers rather than babies

For caregivers rather than babies

For caregivers rather than babies

Even if it's technically safe, there's a fear that it could be harmful to babies. electromagnetic radiation fears that could harm babies.

Even if it's technically safe, there's a fear that it could be harmful to babies. electromagnetic radiation fears that could harm babies.

Even if it's technically safe, there's a fear that it could be harmful to babies. electromagnetic radiation fears that could harm babies.

Worn by baby

Worn by baby

Worn by baby

Worn by parent

Worn by parent

Worn by parent

Ideate

Ideate
Ideate

Added a mobile app to record cries and track long-term trends.

Added a mobile app to record cries and track long-term trends.

Added a mobile app to record cries and track long-term trends.

Recording

Recording

Recording

Monitoring

Monitoring

Monitoring

Interaction Design

Interaction Design
Interaction Design

Considered interaction between wearable device and smartphone.

Considered interaction between wearable device and smartphone.

Considered interaction between wearable device and smartphone.

Service Modeling

Service Modeling
Service Modeling

Instant response builds trust in care.

Instant response builds trust in care.

Instant response builds trust in care.

Guideline

Guideline
Guideline

A wearable captures and analyzes crying sounds.

A wearable captures and analyzes crying sounds.

A wearable captures and analyzes crying sounds.

Guidance is shown as speech bubbles, as if the baby is talking.

Guidance is shown as speech bubbles, as if the baby is talking.

Hungry

Hungry

Hungry

Approaching feeding time

Approaching feeding time

Approaching feeding time

Fasting discomfort

Fasting discomfort

Fasting discomfort

Annoyed

Annoyed

Annoyed

Hyperarousal

Hyperarousal

Hyperarousal

Sudden changes

Sudden changes

Sudden changes

No burp

No burp

No burp

Pain

Pain

Pain

Temperature discomfort

Temperature discomfort

Temperature discomfort

Posture discomfort

Posture discomfort

Posture discomfort

Sleepy

Sleepy

Sleepy

Wake in sleep

Wake in sleep

Wake in sleep

Sleep desire

Sleep desire

Sleep desire

Bored

Bored

Bored

Attention/Attachment Needs

Attention/Attachment Needs

Attention/Attachment Needs

Lack of visual stimulation

Lack of visual stimulation

Lack of visual stimulation

Unsettled

Unsettled

Unsettled

Diaper wetting

Diaper wetting

Diaper wetting

Tickling

Tickling

Tickling

*Source: The study “Emotional Perception of Infant Crying by Expectant Couples”

*Source: The study “Emotional Perception of Infant Crying by Expectant Couples”

*Source: The study “Emotional Perception of Infant Crying by Expectant Couples”

Usability Test

Usability Test
Usability Test

Improving cry detection with a wearable necklace device.

Improving cry detection with a wearable necklace device.

Improving cry detection with a wearable necklace device.

[How the product works]

[How the product works]

[How the product works]

Microphone

Microphone

Microphone

PC

PC

PC

Teachable machine

Teachable machine

Teachable machine

Text data

Text data

Text data

Display

Display

Display

p5.js

p5.js

p5.js

Arduino

Arduino

Arduino

Sound

(Sample)

Sound

(Sample)

Sound

(Sample)

‘Hungry’

‘Hungry’

‘Hungry’

Hungry

Hungry

Hungry

🎙

🎙

🎙

🔈

🔈

🔈

💻

💻

💻

[Secondary Product Development]

[Secondary Product Development]

[Secondary Product Development]

Proceed with physical mockups, test prototypes.

Proceed with physical mockups, test prototypes.

Proceed with physical mockups, test prototypes.

1:1 scale prototyping

1:1 scale prototyping

1:1 scale prototyping

Feature

Feature
Feature

Turning cries into meaning, closing the distance between hearts.

Turning cries into meaning, closing the distance between hearts.

Turning cries into meaning, closing the distance between hearts.

Waveform-based cry analysis reveals the baby’s condition and displays the reason on the device screen.

Waveform-based cry analysis reveals the baby’s condition and displays the reason on the device screen.

Waveform-based cry analysis reveals the baby’s condition and displays the reason on the device screen.

Initial Ideation

Initial Ideation
Initial Ideation

Users said the hierarchy is confusing and needs more explanation.

Users said the hierarchy is confusing and needs more explanation.

Users said the hierarchy is confusing and needs more explanation.

“Even within the 4 categories, the reasons for crying should be more detailed.”

“Even within the 4 categories, the reasons for crying should be more detailed.”

“Even within the 4 categories, the reasons for crying should be more detailed.”

“It’s unclear when exactly the baby will cry tomorrow.”

“It’s unclear when exactly the baby will cry tomorrow.”

“It’s unclear when exactly the baby will cry tomorrow.”

Inspirations from Other Services

Inspirations from Other Services
Inspirations from Other Services

Caregivers can respond to a baby’s cry with positivity.

Caregivers can respond to a baby’s cry with positivity.

Caregivers can respond to a baby’s cry with positivity.

Crying patterns at a glance

Crying patterns at a glance

Crying patterns at a glance

Source : Period app ‘clue’

Source : Period app ‘clue’

Source : Period app ‘clue’

Compared a baby’s cry to a bird’s song

Compared a baby’s cry to a bird’s song

Compared a baby’s cry to a bird’s song

Types of birds the baby collected

Types of birds the baby collected

Types of birds the baby collected

Sulfur-crested Cockatoo

Sulfur-crested Cockatoo

Sulfur-crested Cockatoo

Find out which bird your call sounds like

Find out which bird your call sounds like

Find out which bird your call sounds like

Adaptability

Adaptability

Adaptability

Cutie

Cutie

Cutie

Liveliness

Liveliness

Liveliness

Energy

Energy

Energy

Lu

Lu

Lu

Specialness

Specialness

Specialness

Source : Crying Bebe

Source : Crying Bebe

Source : Crying Bebe

Using Notes for Insights

Using Notes for Insights

Using Notes for Insights

Baby Cry Reason

Baby Cry Reason

Baby Cry Reason

Hungry

Hungry

Hungry

Sensitive

Sensitive

Sensitive

Memo about ‘What’s happening’

Memo about ‘What’s happening’

Memo about ‘What’s happening’

Sleepy

Sleepy

Sleepy

Source : Crying Bebe

Source : Crying Bebe

Source : Crying Bebe

Direction

Direction
Direction

UI updates in real time with changes and explains reasons in detail.

UI updates in real time with changes and explains reasons in detail.

UI updates in real time with changes and explains reasons in detail.

Final Design

Final Design
Final Design

Identifying specific cause, giving solution,

and predicting crying for effective care.

Identifying specific cause, giving solution,

and predicting crying for effective care.

Identifying specific cause, giving solution,

and predicting crying for effective care.

By analyzing the frequency patterns of crying, cycle and daily/monthly reports offer insights into

the baby’s condition—empowering parents with a deeper understanding for better care.

By analyzing the frequency patterns of crying, cycle and daily/monthly reports offer insights into

the baby’s condition—empowering parents with a deeper understanding for better care.

By analyzing the frequency patterns of crying, cycle and daily/monthly reports offer insights into

the baby’s condition—empowering parents with a deeper understanding for better care.

Feature

Feature
Feature

Track history through scientific cry records.

Track history through scientific cry records.

Track history through scientific cry records.

Cry pattern insights offer reassurance and help caregivers feel confident about their baby’s growth.

Cry pattern insights offer reassurance and help caregivers feel confident about their baby’s growth.

Cry pattern insights offer reassurance and help caregivers feel confident about their baby’s growth.

Crying time per day

Crying time per day

Crying time per day

Crying Pattern

Crying Pattern

Crying Pattern

울음 빈도

울음 빈도

울음 빈도

My baby is easy going

My baby is easy going

My baby is easy going

Other babies cry a lot at 9-10am, and my baby is more sensitive at night than during the day, so he cries a lot at 8pm.

Other babies cry a lot at 9-10am, and my baby is more sensitive at night than during the day, so he cries a lot at 8pm.

Other babies cry a lot at 9-10am, and my baby is more sensitive at night than during the day, so he cries a lot at 8pm.

My baby

My baby

My baby

Other Baby(Average)

Other Baby(Average)

Other Baby(Average)

Comparing with peers

Comparing with peers

Comparing with peers

3H 13M

3H 13M

3H 13M

😀

😀

😀

5H 17M

5H 17M

5H 17M

Last week

Last week

Last week

Feel reassured through statistical comparison.

Feel reassured through statistical comparison.

Feel reassured through statistical comparison.

Growth anxiety meets reassurance through clear comparison.

Growth anxiety meets reassurance through clear comparison.

Growth anxiety meets reassurance through clear comparison.

Knowing that other babies cry often at this age really reassures me.

Knowing that other babies cry often at this age really reassures me.

Knowing that other babies cry often at this age really reassures me.

User Feedback

User Feedback

User Feedback

*2024.02.10 / 3 People

*2024.02.10 / 3 People

*2024.02.10 / 3 People

There's a change!

There's a change!

There's a change!

His crying pattern has changed, but that's okay, he's thriving!

His crying pattern has changed, but that's okay, he's thriving!

His crying pattern has changed, but that's okay, he's thriving!

Cry cycle

Cry cycle

Cry cycle

Cry pattern changes

Cry pattern changes

Cry pattern changes

Typical

Typical

Typical

Typical

Typical

Typical

3 Hours

3 Hours

3 Hours

8 Days

8 Days

8 Days

Real-time tracking, even as patterns change.

Real-time tracking, even as patterns change.

Real-time tracking, even as patterns change.

Adaptive Monitoring for Evolving Cry Patterns

Adaptive Monitoring for Evolving Cry Patterns

Adaptive Monitoring for Evolving Cry Patterns

It’s really convenient how it monitors

every time there’s a change.

It’s really convenient how it monitors

every time there’s a change.

It’s really convenient how it monitors

every time there’s a change.

User Feedback

User Feedback

User Feedback

*2024.02.10 / 3 People

*2024.02.10 / 3 People

*2024.02.10 / 3 People

Hungry

Hungry

Hungry

Discomfort

Discomfort

Discomfort

Sleepy

Sleepy

Sleepy

Bored

Bored

Bored

찝찝함

찝찝함

찝찝함

Feeding them cold formula works well.

Feeding them cold formula works well.

Feeding them cold formula works well.

Note-taking keywords

Note-taking keywords

Note-taking keywords

Adjusting the feeding experience

Adjusting the feeding experience

Adjusting the feeding experience

61%

61%

61%

39%

39%

39%

Adjusting the feeding experience

Adjusting the feeding experience

Adjusting the feeding experience

5 minutes on the back

5 minutes on the back

5 minutes on the back

Low temperature formula

Low temperature formula

Low temperature formula

Split feeding

Split feeding

Split feeding

21 cases

21 cases

21 cases

14 cases

14 cases

14 cases

8 cases

8 cases

8 cases

2 cases

2 cases

2 cases

Last week

Last week

Last week

Effective solutions, tailored to my baby.

Effective solutions, tailored to my baby.

Effective solutions, tailored to my baby.

My baby’s unique traits, all at a glance—no need to check old memos.

My baby’s unique traits, all at a glance—no need to check old memos.

My baby’s unique traits, all at a glance—no need to check old memos.

I used to struggle to find what I had recorded — now it’s all in one clear view.

I used to struggle to find what I had recorded — now it’s all in one clear view.

I used to struggle to find what I had recorded — now it’s all in one clear view.

User Feedback

User Feedback

User Feedback

*2024.02.10 / 3 People

*2024.02.10 / 3 People

*2024.02.10 / 3 People

Accessibility Validation

Accessibility Validation
Accessibility Validation

Grasp at a glance, even while doing housework.

Grasp at a glance, even while doing housework.

Grasp at a glance, even while doing housework.

Reflections

Reflections
Reflections

What I learned

What I learned

What I learned

Empathy before data.

In sensitive situations like a baby’s cry, users don’t analyze, they feel.

I learned that trust comes not from detailed explanations, but from delivering a sense of emotional safety. The product should start with reassurance, not precision.

Empathy before data.

In sensitive situations like a baby’s cry, users don’t analyze, they feel.

I learned that trust comes not from detailed explanations, but from delivering a sense of emotional safety. The product should start with reassurance, not precision.

Empathy before data.

In sensitive situations like a baby’s cry, users don’t analyze, they feel.

I learned that trust comes not from detailed explanations, but from delivering a sense of emotional safety. The product should start with reassurance, not precision.

What Could Be Improved

What Could Be Improved
What Could Be Improved

Rather than simply labeling emotions, the product leverages AI to deliver clear, actionable guidance that helps in the moment. To enable this, the system is powered up through AI and requires further iteration using real-world data and contextual UX research.

Rather than simply labeling emotions, the product leverages AI to deliver clear, actionable guidance that helps in the moment. To enable this, the system is powered up through AI and requires further iteration using real-world data and contextual UX research.

Rather than simply labeling emotions, the product leverages AI to deliver clear, actionable guidance that helps in the moment. To enable this, the system is powered up through AI and requires further iteration using real-world data and contextual UX research.